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outcomes. Each year Tennessee *s State Board of Education produces a 
report card on each school district using data provided by the 
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two report card sections: testing information (grade level, the prior 
school level, the present school year, and state average) and system 
information (professional educator information and student 
information organized under five headings: grade level, 2 years prior 
data, present year, and state average). Using 1988-89 data, eight 
categories of report card information were considered as follow: (1) 
county par capita income; (2) average professional salary; (3) 
expenditures per pupil; (4) average daily membership; (5) percent of 
students in attendance; (6) percent of oversized classes; (7) percent 
of students on free or reduced lunches; and (8) percent of 
professionals on Levels II and III of Tennessee*s Career Ladder 
program. A state composite profile based on 139 districts was 
compared to report cards of 121 districts constituting the final 
sample. The relationships between these characteristics and student 
achievement outcomes were examined, and districts were compared. 
Analyses indicated that these eight variables do not explain student 
outcomes. Study results suggest that the report cards fail to reflect 
much of what accounts for student achievement. There are nine 
references and 12 tables. Four appendixes provide 26 figures that 
present supplemental information on linear regression, partial 
correlation, truncated linear analysis, and polynomial regression 
analyses. (SLD) 
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WHAT POLICYMAKERS CAN LEARN FROM SCHOOL REPORT CARDS: 
ANALYSIS OF TENNESSEE'S REPORT CARDS ON SCHOOLS. 

I. INTRODUCTION 

Currently, there is great interest in accountability based Oii student perfomDance. "Report cards" on schools 
have become common in many states, and that concept is imbedded in the national education agenda 
(America 2000). However, little attention has been given to the value and uses of "report card" data. 

In Tennessee (TN), a report card developed for each school district contains data on student academic 
achievement (test scores), educational expenditures, community per capita income, student enrollments and 
attendance, professional personnel and salaries, average class sizes and other factors deemed important to 
understanding education in the community and in providing comparisons with other schools or districts in the 
state and region. Generally, a reader does not know what relationship the entries have to each other or to 
student achievement, which is considered the 'tjottom line." The inclusions tend to imply that the factors 
reported strongly influence student achievement but that relationship is not directly stated or explored. 

The puipose of this research was o study the relationships among factors reported on one set of report 
cards— 4hose developed for bcal school districts within TN. The analysis offers information about how the 
selected factors relate to student outcomes. Results should interest policymakers and educators as they attempt 
to detemnine where and how to allocate resources (nnoney, personnel, etc.), and as they consider report cards. 

II. TENNESSEE'S (TN) REPORT CARD (1988-1989) 

Each year, Tennessee's State Board of Education produces a report card on each school district using data 
provided by the district and by the State Department of BJucation. The cards report district-level data; i.e., 
individual school data are not reported. Tables 1 and 2 display the "WkJget" city (TN) school district report card. 
Data are organized under two sections: (1) Testing infomiation, and (2) System information. System 
Information includes professional educator information and student irtformation. 

Testing infomnalion for four sets of tests (in 1 988-89) is organized under four headings: grade level, the 
prior school year (1987-88), the present school year (1988-89), and state average. System information is 
organized under five headings: grade level, two years prior data (1986-87) and(1987-88), present year (1988- 
89), and state average. Testing infonnation is defined as follows (with abbreviations used in this paper): 

a. Basic Skills First Achievement Test (BSF) The BSF is a staie<leveloped cnterion-referenced test 
administered to students in grades 3, 6, and 8 to n)easure how well students have mastered reading 
and mathematics skills. Scores show the average percent of skills mastered by students in system. 
This study used the 1989. Sth^grade data. 

b. Stanford Achievement Test (SAT) The SAT was administered in grades 2, 5, 7 and reported in 
stanines. The SAT was discontinued (1989) and replaced by the STAS (Task 2). This study did not 
use the now discontinued SAT in the analyses. 

1 
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Table 1 Testing information for Widget City Scliools. Similar achievement data are 
found on the typical Tennessee School districts Report Card. 



Testing Information 
for Widget City 


Grade 
Level 


1987-88 


1988-89 


State 
Average 


Basic Sl(iils First 
Achievement Test 

(percent score) 


Reading 


3 


90 


88 


80 


6 


82 


80 


77 


8 


92 


91 


81 


Math 


3 


91 


90 


82 


6 


67 


71 


66 


8 


77 


84 


66 


Stanford 
Test 

(Stanine score) 

7-9 = High 
4-6 « Average 
1-3«Low 


Reading 


2 


6 


7 


6 


5 


6 


6 


5 


7 


6 


6 


5 


Math 


2 


7 


8 


6 


5 


7 


6 


6 


7 


7 


7 


5 


Spelling 


2 


6 


7 


6 


Language 


5 


6 


6 


5 


7 


6 


6 


5 


Environment 


2 


7 


7 


6 


Science 


5 


6 


7 


6 


7 


6 


6 


5 


Listening 


2 


7 


7 


5 


5 


6 


6 


5 


7 


6 


6 


5 


Social 
Science 


5 


6 


6 


5 


7 


6 


6 


5 


Stanford Test of 
Academic Skills 

(TASK 2) 

7-9 = High 
4-6 = Average 
1-3= Low 


Reading 


9 


6 


6 


5 


12 


6 


6 


5 


Math 


9 


6 


6 


5 


12 


6 


6 


5 


English 


ft 


7 


6 


5 


12 


6 


7 


5 


Science 


9 


7 


6 


5 


12 


6 


6 


5 


Social 
Science 


9 


5 


6 


5 


12 


6 


5 


5 


Tennessee Proficiency Test 

(% Students Passing) 


Language 


9 


88 


92 


78 


Math 


9 


95 


98 


90 


Both 


9 


86 


91 


76 
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Table 2. General information fc und on a typical school district's report card. 



Svstem Information 

V & III ■ 1 i 1 \0 1 ill %4 Wt i \^ 1 1 

for Widget City 


Grade 
Level 


1 986-87 

1 sou Of 


1 987-88 


1 Q8a-8Q 

1 S700 0«7 




Number of Schools 


K-12 


5 


5 


5 


12 


Average Daily Mombership 


K-12 


3,291 


3,394 


3,372 


5,874 


% Student Attendance 


K-12 


95.7 


95.3 


95.1 


93.6 


% Enrollment Chanye 


9-12 


-13.0 


-16.1 


-15.2 


-24.7 


% Oversized Class 


K-12 


1.2 


1.4 


2.3 


3.8 


% Of Students on Free or Reduced Price Lunch 


K-12 


23 


21 


21 


42 


F- vr\a nrJiti tree rtar mmil 

CApoiiQiiurob poi pupil 


iS-l d. 


$2,71 8 


$3,299 


$3,501 


$3,304 


County Per Capita Income 








$12,819 


$12,878 


% Elementary Schools Accredited by SAGS 


K-o 




100.0 


100.0 


29.1 


% Secondary Schools Accredited by SACS 


/ -1 ii 


100.0 


100.0 


100.0 


64.9 


Professional Educator Infonnation 












% Professionals on Career Ladder Levels II & III 


K-12 


22.9 


21.9 


25.6 


14.8 


Average Professional Salary 


K-12 


$25,198.60 


$26,085.44 


$30,804.37 


$26,756 


Student Infonnatlon 












% Diplomas 
Granted 


Regular 


12 


90.6 


GS 7 


75.8 


81 8 


Honors 


12 


49.6 


26.7 


20.0 


8.5 


Special Education 


12 


1.8 


1.4 


1.5 


1.9 


Certificate of Attendance 


12 








0.9 


Seniors not Receiving 
Diploma in Spring 
Graduation 


12 


2.7 


3.2 


2.7 


6.9 


% Students In Vocational Education Courses 


7-12 


33.0 


40.9 


41 .0 


45.5 


% Students in Special Education 


K-12 


12.1 


11.3 


12.1 


14.2 


% Chapter 1 Students 


K-12 


13.3 


15.5 


12.4 


11.9 



c. Stanford Test of AcacJemIc Skills (TASK 2) (STAS) The STAS Is administered t in grades 2, 5, 7, 9, 
and 12, These tests show how students perform when compared to students at the same grade 
nationwide. These scores are reported as stantnes, scores from one to nine, where one is low and nine 
is high. An average score is four, Tfve, or six. These scores show the average stanine on each part of 
the test for the students in the system. This study usecj the 1989. 12th-qrade data. 

d. Tennessee Proficiency Test (TPT) The TPT measures minimum skills in mathematics and language 
arts. Students must achieve a passing score of 70 percent correct on both the math and language arts 
tests in order to fulfill one of the requirements for receiving a regular diploma. Students take the test for 
the first time in the ninth grade. This study used the 1989. Qth-grade data. 

Seven of the eight categories of report card information used for this study are defined by report card 

developers (Average Daily Membershipwas not specifically defined): The definitions are: 

a. County per Capita iirtcome (CCI) This figure represents the per capita personal income for the county 
in which your school system is located. The most recent figures available from the U.S. Bureau of 
Economic Analysis are for 1987. 

b. Average Professional Salary (APS) 777/s figure shows the estimated average salary for all 
certificated personnel in your school system, 

c. Expenditures per Pupil (EPP) This figure shows the average number of dollars spent for each pupil in 
average daily attendance for your school system, 

d. Average Daily Memberslifp (ADM) ( The Report Card did not provide a definition for this category}. 

e. Percent of Students In Attendance (%SA) This figure shows the average percent of students in 
attendancedaily in the individual school districts for the school years 1986-87, 1987-88, 1988-89, 

I Percent of Oversized Classes {%00) This figure shows the percent of classes in all grade levels 
which had waivers for being over the maximum class size. Maximum class sizes in Tennessee are: 25 
for grades K-3, 28 for grade 4, 30 for grades 5-6, 35 for grades 7-12, and 23 for vocational (1989). 

g. Percent of Students on Free or Reduced Lunches (%FRL) Students whose family income meets 
certain criteria are eligible for free or reduced price lunches. This figure shows the percent of students 
in your school system who receive free or reduced price lunches, 

h. Percent Professionals on Career Ladder Levels II and III (%CL) This figure shows the 
percent of professional staff in your school system who have met the standards for Career Levels 
II and III. These are the upper rungs of Tennessee's Career Ladder program. The number 
includes regular classroom teachers, guidance counselors, librarians and administrators. 

The investigators chose the above eight categories {including ADM) and test results for detailed study. 

Many educators and lay persons belie ye that these factors influence student acadeniic performance. 



III. METHODOLOGY 

Investigators used the 1 988-89 report card data for the study. Although the report cards provided test 
results at several grade levels, linl^ li^e average scores for Hog highest grade leve ls reported were used in these 
analyses . Those scores were considered as best representing the final product of the school system. Seven 
research questions guided the study: 

1 . How do school district characteristics currently reported in the report cards relate to reported student 
academic achievement? 

2. How do reported school district characteristics relate to each other? 

3. When rank ordered on the basis of student outconr)es, how do school districis within the state perform in 
terms of the district and community characteristics reported in the report card? 

4 
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4. Do the reported school district cliaracterislics appear to represent all or most factors that influerx» 
student academic achievemerit? 

5. When academic achievement is treated as scores on three separate test batteries, are patterns of 
influence changed? 

6. Do "outlier** school districts significantly influence the findings of this study? 

7. How might the findings of this study inform educational policy at state and local levels? 

Investigators treated student outcome data (test data) as the dependent variable and other characteristics 

reported as independent variables that influence student outcomes. Several analyses were conducted. 

> 

First, a conDposite profile of all school districts in the state (n=1 39) was produced, as was a profile of the 1 21 
school districts constituting the final sanrple. These profiles provided a statewide picture and helped 
researchers to determine if the study sample (n=121) varied in any significant way from the state composite. 

To answer research question #1 , three analyses were conducted. Rrst, a linear regression assessed the 
relationship between each reported characteristic and the school district's mean student outcome. The 
researchers combined three sets of reported test scores to create a mean student outcome (MSO) rr^easure for 
each school district. To create the MSO, the mean scores reported for each test were first converted to z- 
scores (see Nunnally , 1 978, pp 24-34) that were used to confute the MSO (As shown later, the MSO was a 
more useful variable than any single testing outcome). Only school districts (n=1 21 ) reporting scores for all 
three tests (8th grade BSF; 1 2th grade STAS; and 9th grade TPT) were in the final sample. A Pearson Product 
Moment correlation was developed as a means of comparing categories. 

The second analysis for question #1 used two procedures. First, report card information was reorganized 
into three clusters: testing, nrwney, and school system information. That staicture is outlined below: 

1 . Testing information (combined into the MSO) 

• Basic Skills First Achievement Test (BSF) 

• Stanford Test of Academic Skills (Task 2) (STAS) 

• Tennessee Proficiency Test (TPT) 

2. Money Information 

• County per Capita Inuome (CCl) 

• Average Professional Salary (APS) 

• Expenditure Per Pupil (EPP) 

3. School System Information 

• Average Daily Membership (ADM) 

• Percent of Students in Attendance (%SA) 

• Percent of Oversized Class (%0C) 

• Percent of Students on Free or Reduced Lunches (%FRL) 

• Percent of Professionals on Career Ladder Levels II and 111 (%CL) 

The purpose of this clustering was to facilitate investigation of the interactions of multiple, related categories 
with student outcomes and with each other. Data for categories clustered as "Money" or "School System" were 
converted to z-scores, then the Pearson Product Morrient correlation was applied. 

5 



The third analysis employed Guttmarfs partial correlation procedure. The partial correlations for each category 
were converted to percentages of influence, thereby enabling investigators to determine the relationship of each 
independent variable (category) to the dependent variable (MSO). The procedure was then reperJed using three 
dependent variables [scores on the separate outcomes measures (.e., BSF, STAS, and TPT)]. 

To answer research question #2, investigators corrputed correlations among independent variables 
(categories in the Money and School System clusters). A coefficient of determination (r^) showed the levels of 
interaction between categories within a cluster and across clusters. 

Research question #3 required the rank ordering of districts within the sample by MSO, The investigators 
then considered the nine districts with the highest MSOs and the nine with the lowest MSOs. 

Research question #4 required no further statistical analyses. The partial correlation coefficients and 
related percentages of influence previously developed provkjed the necessary data. 

To respond to research question #5, the Guttman partial conrelatfon procedure was applied to the relationship 
between each independent variable and each of the three test scores available for each school district. 

Research question #6 allowed investigators to examine the effects of "outlier districts on the statewide 
profile of relationships among variables. To respond to this question, the top 10% aM tx)ttom 10% of the 
school districts in each independent variable (student attendance, oversized classes, etc.) were eliminated fnom 
consideration, and a new Pearson Product Moment conelation coefficient for each independent variable and 
MSO was computed. Fishers Zr-transfonriation was then used to evaluate ttie significance of the relationship 
between the two correlation coefficients (MSO and Truncated MSO). A second test was also applied to the 
existing data. A third order cubic polynomial was computed for each relationship; i.e., interaction between 
independent variable and MSO. Then Fisher's Zr-transfonration was used to determine the significance 
between the original conrelation coefficient and the new, polynomial coefficient. 

Research question #7 was a means of focusing conclusions and implications. The development of report 
cards on schools and the demographic data reported in them reflect unstated assumptions about their 
relationship to a great extent. If the findings of this study have any value, it may well be the informing of 
policymakers and policy. 

iV. FINDINGS 

Findings of the study are reported in two areas: (A) descriptive analyses of school districts and (B) 
responses to the research questions. 



A. Descriptive Analyses Of School Districts 



1. The 1988-89 Prof lie of TN's 139 School Districts 

A profile of TN school districts (n=139) by Report Card category was developed (see Table 3). For each 
category, the report card (state) mean score, standard deviation (SD)> number of schools submitting data and 
ranges of scores or numbers were compiled. 

a. Outcome Data 

Of the 139 TN school districts, 96% reported BSF scores, 89% reported TPT scores, and 87% reported 
STAS scores. An average of 81 % of ail students taking the 8th grade BSF in 1 988-89 passed the Reading 
sutj-test, and 66% passed the mathematics component. Just over 76% of all students completing the TPT 



Table 3. A "Report Card" Profile of Tennessee's 1 39 School Districts in 1 988-89. 



139 SCHOOL DISTRICTS 



Report Card 
SD. n Max Min. Mean 



Basic Skills First (BSF) 

Reading 4.9 

Math 7.6 

Stanford (STAS); Task 2 

Reading .5 

Math .5 

English .6 

Science .5 

Social Studies .5 

TN Proficiency Test (TPT) 

Language 7.1 

Math 6.4 

Both 9.4 

MONEY 

Co./Gapita Income ($) (CCI) 2,030 

Expenditure Per Pupil ($) EPP) 505 

Aver. Prof. Salary ($) (APS) 2,683 

SCHOOL SYSTEM 

Average Daily Mem. (#) (ADM) 1 1 ,702 

Student Attendance (%SA) 1 .3 

Oversized Class (%0C) 4.6 

Free/Reduced Lunch (%FRL) 4.3 

Career Ladder ll/lll (%CL) 6.1 



(Percent passing^: 8th grade 
134 93 65 81 

134 85 43 66 

(Stanine scored 12th grade 
121 7 4 5 

121 6 4 5 

121 7 4 5 

121 6 3 5 

121 6 4 5 

(% Students passinok 9th grade 
124 98 56 76 

124 100 59 90 

124 98 48 76 



139 19,318 6,934 12,878 
139 4,891 2,163 3,304 
139 34,796 21,015 26,756 



139 104,788 188 5,874 

139 97.1 90.3 93.6 

122 21.5 0.1 3.8 

139 86.0 9.0 42.5 

138 41.5 4.1 14.8 



passed both the mathematics and language components. More students passed the TPT mathematics sub- 
test (90%) than the Language component (76%), a rever::al of the results on the BSF, usually given a year 
earlier. In 1 988-^9, the mean scores of 1 2th grade students taking the STAS was near the 5th stanine (4.8-5.3) 
on all five sub-tests. Scores on all tests ranged from several SDs below to several above the mean. 

b. lUloney Matters 

In the Student Expendtture Per Year category, students in the richest school district were supported by 
$4,891 per year and the students in the poorest district by $2,163 per year, a $2,728 per year difference 
between the richest and poorest districts . The mean County per Capita Income was $1 2,878« with a range 
from $^9,3^&\o $6,934 (an approximate 300% difference). The Average Professional Salary meanwas 
$26,756, with a range from $34,796 to $21 ,015. 

c. School System characteristics 

All TN districts provided Average Daily Membership (ADM) data The average school district's daily 
attendance was 5,874 students. The largest district had 104,778 students and the smallest had 188 students. 
The percent of Student Attendance (7oSA) for the top school district in this category was two SDs above the 
mean and for the bottom school district, two SDs below the mean. The top district had 97% attendance and the 
lowest district had 90% atterxlance— a 7% difference. Percent of oversized classes (%0C) was reported by 
122 school districts. State class size restrictions in 1988-89 were 25 students in grades K-3, 28 students in 
grade 4, 30 students in grades 5-6, 35 students in grades 7-12, and 23 students in vocational classes. An 
oversized class was any class exceeding these nu-nbers. The percentage of oversized classes reported 
ranged from 0.1% to 21 .5%. 

Students participating in the Free or Reduced Lunch (%FRL) program in 1988-89 ranged from a low of 
9% to a high of 86%. This category is one con^Don (but usually understated) indicator of the socio-economic 
status (SES) level of families served by the school. 

Tennessee has an operational career ladder program for professional educators (teachers, adnrtinistrators, 
other). Since 1984,educators have been able to apply for evaluation for levels I, II, and III on the Career Ladder 
and receive merit pay as well as extended contract opportunities, if they achieve the upper mngs (Levels II, III). 
In 1988-89, approximately 15% of all educators had achieved Career Levels II and III (%CL). Percentages of 
educators at these levels in individual school districts ranged from 4% to 42%. 

2. Descriptive Analysis of 1 21 TN School Districts Reporting Necessary Student Outcome Data 

Student outcomes were a major focus of this study, so school districts that did rwt report all three sets of 

test scores were excluded from further investigation. Of 1 39 school districts, 1 21 provided all necessary student 
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test data. The profile for these 121 districts was like that of Vne total 139 districts (see Table 4). Mean 
County/district per capita income dropped 75 dollars. Average expenditure per pupil rose 19 dollars. Average 
professional salary fell 84 dollars. Average menribershlp rose 689 students. Oversized classes dropped 0.4%. 
Students in the free or reduced lunch program rose 0.5%, and career levels II and III teachers dropped 02 %. 

B. Findings Pertinent to Researoh Quessions (All correlations appear In the correlation matrix Jable 5) 

1 . How do school district characteristics curremiy reported in the report cards relate to reported 
student achievement? 

A linear regression model was used to assess the relationship between each reported characteristic and 
the school district's mean student outcome (MSO). !n addition, by using the state mean and MSO, outlier 
school districts were identified. Below are the results of this analysis reported by characteristic. 

Table 4. A Report Card Profile of 121 Tennessee School Districts sampled. 



121 SCHOOL DISTRICTS 







n 


lUlax 


Min. 


Report Card 
■Mean 


OUTCOMES 












Basic Skills First (BSF) 






(Percent passing^ 8th grade 


Reading 


4.9 


121 


91 


65 


81 


Math 


7.7 


121 


85 


43 


66 


Stanford (STAS); Task 2 






(Stanine score); 12th arade 


Reading 


0.5 


121 


7 


4 


5 


Matti 


0.5 


121 


6 


4 


5 


Englisti 


0.6 


121 


7 


4 


5 


Science 


0.5 


121 


6 


3 


5 


Social Studies 


0.5 


121 


6 


4 


5 


TN Proficiency Test 






(% students Passi 


na):9th orade 


Language 


8.6 


121 


98 


56 


76 


Matli 


6.4 


121 


98 


59 


90 


Both 


9.3 


121 


98 


48 


76 


imONEY 












Cc ./Capita Income ($)(CCI) 


1,962 


121 


19,318 


6,934 


12,878 


Stud. Expenditure ($)(EPP) 


509 


121 


4,891 


2,318 


3.304 


Aver. Prof. Salary ($) (APS 


2,693 


121 


34,797 


21,286 


26,756 


SCHOOL SYSTEM 












Average Daily Mem.(#) (ADM) 


12,395 


121 


104,788 


375 


5,87 4 


Student Attendance |%SA) 


1.3 


121 


97.1 


90.3 


93.6 


Oversized Class (%0C) 


4.1 


110 


21.5 


0.1 


3.8 


Free/Reduced Lunch (%FRL) 


14.5 


121 


86.0 


9.0 


42.0 


Career Ladder ll/lil (%CL) 


5.9 


121 


41.5 


4.1 


14.8 
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Table 5. Correlation Matrix for testing, money, and sclwol system categories (Pearson). 



Outcome 



Money 



School System 





BSF 


STAS 


TPT 


MSO 


CCI 


APS 


EPP 


ADM 


%SA 


%0C 


%FRL 


BSF 


1 






















STAS 


.358 


1 




















TPT 


.387 


.438 


1 


















MSO 


.753 


.r/0 


.793 


1 
















CCI 


.140 


.368 


.219 


.313 


1 














APS 


.214 


.528 


.272 


.436 


.706 


1 












EPP 


.081 


.354 


.087 


.223 


.437 


.783 


1 










ADM 


-.146 


-.000 


-.061 


-.090 


.529 


.410 


.330 


1 








%SA 


.403 


.117 


.418 


.407 


.063 


.042 


-.138 


-.155 


1 






%0C 


-.235 


-.294 


-.174 


-.302 


-.356 


-.458 


-.390 


-.198 


-.203 


1 




%FRL 


-.347 


-.422 


-.362 


-.488 


-.533 


-.372 


-.003 


.063 


-.259 


.191 


1 


%CL 


.188 


.317 


.214 


.310 


.338 


.468 


.266 


.025 


.209 


-.348 


-.321 



£.01 =Bold 



a. Influence of County Per Capita Income (CCI) 

Figure 1 (p. 11 , and Appendix A, p. 29) shows a positive correlation t)etween MSO and CCI. The 
regression line intercepts the MSO zero z-score mean at $1 1 ,000, a point $1 ,800 below the state average per 
capita income of $1 2,878. Generally, districts with greater CCI demonstrated higher MSO (r=^,31 ). However, 
most (i.e., »9:1 ratio) school districts' per capita income falls below the reported state mean ($1 2,878). 

Even while a positive relationship between CCI and MSO was denx)nstrated, there were "outlier" districts. 
Four districts had CCIs at least $2,000 below the state mean, but had MSOs at least one standard deviation 
above the mean. One of these districts had a per capita income almost $6,000 below that mean. 

b. Influence of Average Professional Salary (APS) 

The significant positive correlation between size of APS and MSO (r=.44) was the highest positive 
correlation for the three mone*' categories. The mean APS in 1 988-89 was $26,756, with a range from $21 ,000 
to $35,000 (see Figure 2, p. 29). Most school districts («4:1 ratio) have a smaller APS than the state mean of 
$26,756. 

Generally, school districts with higher APS demonstrated higher MSO's. However, this category also 
displayed "outlier" school distncts. One of the nine districts with the highest MSOs in the state had an APS of 
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County per Capita Income (CCI) 

Figure 1. Relation of Per Capita Incxjme to Mean Student Out(X)me. ^ 

less than $22,000; another's APS was $24,000, and two others were under $25,000. Of the nine districts with 
the lowest MSO z-s(X)res, all but one were below the APS state average. 

c. Influence of Expenditure Per Pupil (EPP). 

The majority of TN school districts did not expend the state mean EPP of $3,306 (=4:1 ratio). The trend 
line in Figure 3 (p. 30) shows a small positive con-elation between MSO and EPP. The EPP congelation (r=.22) 
with MSO was the smallest of the three money category correlations (see Table 5, p 1 0). 

Wliile districts with higher EPP tended to have slightly higher MSOs than other districts, six of ten districts 
with MSOs at or exceeding one SD above the mean were below the state niean in EPP. Two of those districts 
spent at least $500 less than the mean, and four spent at least $400 less. For every TN school district above 
the mean EPP, four were below the EPP mean. 

d. Influence of Average Daily Memberships (ADIUI). 

There was no significant correlation between schcx^l district size (ADM) and MSOs. The slight correlation 
was negative (r= -.09); larger district size tended to influence MSO negatively. 



1 . All other figures (2 through 8) are fomnatted in the same fashion as Figure 1 . They, together with Figure 1 , 
are presented in Appendix A (pp. 29-32). Their inclusion in the text tends to clutter the discussion, and 
they are more meaningful when viewed as a group. 
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The slate mean ADM of 5,874 students per district was influenced by a few large school districts, since 
most districts clustered below that vnean (three districts below for every one above-see Figure 4, p. 30). In 
actuality, 83% of TN school districts reported fewer than 1 5,000 students. Only 1 0 districts had enrollments 
above 10,000 students, and two districts had very large enrollments (over 60,000). 

e. Inf iuence of Student Attendance (%SA) 

A significant, positive correlation (r=--.41 ) existed between %SA and MSO (see Table 5, p. 1 0). Generally, where 
average daily attendance was greater, achievement was higher. 

State average student attendance in 1988-89 was 93.6% (Figure 5, p. 31 ). Most districts exceeded this 
average (ratio of « 7 to 4). All districts where attendance averaged 92% or less demonstrated MSO z-scores at 
or below zero; 6 of 8 districts where average atteixtence exceeded 96% had positive MSO z~scores. 

f . Influence of Oversized Ctass (%0C) 

Of the 121 school districts studied, 1 1 0 reported oversized class data. There was a significant, negative 
correlation between %0C in a district and MSO (r=-.30) (see Table 5, p 10). Generally, districts with larger 
nunt>ers of oversized classes had lower MSOs and vice versa. Low %0C is a high (good) (e.g., 1 , 2) rank. 

ArDong the 1 1 0 school districts reporting %0C, the average percentage of classes exceeding size 
regulations was 3.8 (see Figure 6, p. 31 ). Of the nine school districts with MSO z-score of negati^/e one SD or 
below, seven exceeded the average percentage of oversized classes. Of the 9 school districts with an MSO z- 
score of plus one or ^bove, only 3 exceeded the state average (3.8%). However, 37 districts that reported wore 
than 3.8%OC demonstrated MSOs at or above the state mean. Fifteen districts reporting a %0C below 3,8% 
also had MSO z-scores below zero. The impact of class size cannot be simply interpreted.^ 

g. Influence of Free or Reduced Lunch as Soclo-Economlc Status Indicator (%FRL) 

The percentage of free and reduced lunches (FRL) was one indicator of socio-economic status (SES) of the 
community; the higher %FRL , the lower the SES of the conrvnunity. There was a significant negative correlation 
between %FRL and mean student outcome (r=-.49). School districts with lower percentages of free and reduced 
lunches generally had higher MSO z-scores. High FRL rank (e.g., 1 or 2) is tew %FRL pupils. 

The statewide average %FRL was 42%. However, two-thirds of all school districts reported percentages of 
%FRL below the state average (see Figure 7, p. 32). Seven of the nine districts with MSO z-scores of negative 

2 This is an interesting finding considering the results of Tennessee's massive, four-year tongrtudnal/dass-size 
study, Project STAR, which showed convincing^ the benefits to student achievement of reduced class size 
(to 1 :15) in prinnary grades (Finn & Achiltes, 1990; Word, et, aL, 1990). STAR, however, assessed grades K- 
3; this present analysis reviewed grades 9-12. 
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one SD or below had larger %FRL than the state average, and five of the six school districts v^/ith the highest 
%FRL demonstrated negative MSO z-scores. AHematively, seven of the nine districts with MSO z-scores of 
plus one or higher had lower than state average %FRL. There were"outlier" districts. Two of the nine districts 
with MSO z-scores above plus one v/ere above the state average for %FRL; one had 60% FRL. 

h. Influence of Percentage of Career Ladde^^ (%CL) Teachers 

Since TN's Career Ladder program is intended to identify and to reward outstanding teachers, a high 
percentage of Career Ladder II & III teachers in a school district should indicate good instmction {and high 
MSO) in that district. Low percentages of Career Ladder II & 111 teachers did not necessarily suggest poor 
instruction, because the Career Ladder program is optional. 

There was a significant but modest positive correlation between %CL teachers in a district and MSO 
(r=.31 ). The trend line in Figure 8 (p. 32) suggests that the greater the %CL teachers, the higher the MSO. The 
four school districts with the lowest MSO z-scores also had low %CL teachers. 

The average percentage of upper-level Career Ladder teachers statewide was 14.8%, and slightly more 
districts reported percentages below that average than above (a ==75 ratio). While seven of the ten districts 
generating the highest MSO z-scores reported greater %CL teachers than the state average, there were also 
"outlier'' districts. The district with the highest %CL teachers had a MSO well bselow the state mean. Greater 
%CL of teachers was not a guarantee of better student performance. Keep in mind that Career Ladder status is 
not independent of years of teaching experie»Tce. To be eligible for Career Level II evaluation, teachers need 
eight years of experience; twelve for Career Level 111. 

i. Relationships of the Money and School System £iust^ to Mean Student Outcome (MSO). 

The three Money characteristics (per capita income, per pupil expenditure, and average professional 
salary when combined, produced a significant positive correlation (r=.41 ) with MSO (see Figure 9, p.1 4). This 
finding is not surprising since each characteristic individually provided a positive correlation. Four school 
districts with money z-scores below zero produced MSOs at least one SD above the state mean, and one 
system with a money z-score one and one-half SDs above the mean produced a MSO one and one-half SD 
below the mean. 

Data in Figure 1 0 (p.14) show that the five school system factors (%SA, %FRL, %OC, %CL and ADM), 
when combined, produced a nrxxJerately high, significant positive correlation (r=.50) with MSO. Based on the 
sizes of the correlation coefficients, these school system characteristics seem to influence MSO more than do * 
the reported money factors. This finding suggests that other factors within a school district may be equally or 
more important than nrK)ney. 



High y , . 1 .8386e-5 + 0.35505X R'^Z » 0.167 




Figure 9. Relation of Combined Money Factors to Mean Student Outcome. 



High y= -7.3759e-3 + 0.66884x R'^2 = 0.246 




Z-Score: Mean School System (ADM4'%SA+%OC4-%FRL4-%CL)/5 

Figure 1 0. Relation of School System Factors to Mean Student Outcome. 



Of the eight factors (see Table 5, p. 1 0) studied for their relationships to MSO, five denx)nstrated significant 
(p<.01) positive con-elations, and two fadors yielded significani negative conflations with MSO: 

1 . Average Professional Salary or APS (r=.44) 

2. Percent Student Attendance or %SA (r=-41 ) 

3. County Per Capita Income or CCI (r=^1 ) 

4. Percentage of Career Ladder Teachers II & III or %CL (r=.31) 

5. Expenditure Per Pupil or EPP (r=.22) 

1 . Nuniber (percent) of Free or Reduced Lunches or %FRL (r= -.49) 

2, NunfU)er (percent) of Oversized Classes or %0C (r= -.30) 

Size of School District (Average Daily Membership or ADM) did not show a significant correlation v^ith MSO. 

j. Partial Correlation to try to Isolate Factor Influence. 

These individual correlations do not fully answer research question #1 . To refine the response, Guttman's 
partial correlation procedure was used; i.e., the Ka/s^ was run and a total matrix sampling adequacy greater 
than .500 resulted so that Guttman's assumptions were met. The matrix is in Table 6 (see p.16). 

The partial correlatton coefficients were converted to percentage of influence (PI) as displayed in Table 7 
(see p.1 6). Several trends emerge from this analysis. First, the eight factors investigated account for only 
26.5% of whatever relates to MSO, a fact discussed in wore detail in response to research question #4. 

Of the 26.5% influence on MSO exerted by the combined money and school system variables, alnrwst 1 1% 
was provided by student attendance. A student's school teaming and achievement are influenced by his/her 
presence in school. Six percent of the association with MSO was produced by %FRL; 5,6% was attributable to 
APS. The %0C accounted for only .6% of the MSO results; affluence of the convnunity (CCI) contributed only 
.4% of the influence, and %CL contributed .2%. The EPP alone showed essentially NO association (.003) with 
MSO, Inplications of these findings are discussed in response to research questton #5. 

2. How Do Reported School District Characteristics Interact with Each Other? 

To detemiine the interaction of money and school system factors, they were correlated with each other. As 
denfKinstrated in Table 5 (p 10), significant (p< .01 ) correlations resulted for 1 7 of 28 combinations. 



Significant positive correlations occurred between these nine factors: 




a. CCI and APS 


r=.71 


f . APS and ADM 


r=.41 


b. CCI and EPP 


r=.44 


g. APSand%CL 


r=.47 


c. CCIandADiyi 


r=.53 


h. EPP and ADM 


r=.33 


d. CCIand%CL 


r=.34 


i. EPPand%CL 


r=.27 


e. APS and EPP 


r=.78 







Table 6. Guttman's Partial Correlation matrix for MSO and the other Tennessee 1 989 
Report Card categories. 





Z:MSO Z:CCI 


Z: APS 


Z:EPP 


Z:ADM 


Z:%SA 


Z:%OC 


Z: %FRL 


Z: %CL 


Z: MSO 


.444 


















Z:CCI 




.707 
















Z:APS 




.295 


.839 














Z:EPP 




-.007 


.741 


.735 












Z:ADM 




.506 


.211 


-.151 


.493 










Z:%SA 


,330 


.01 


.005 


-.188 


-.057 


.266 








Z:%OC 


,07e , 


.043 


.033 


.146 


.071 


.176 


.283 






Z: %FRL 


.244. 


.499 


.24 


-.394 


-.391 


-.008 


-.005 


.586 




Z: %CL 


^.040 


.018 


.299 


-.127 


-.166 


.105 


.163 


.024 


.312 



Table 7. Partial correlations and percentage of influence (PI) of each category on mean 
student outcome. 

r r2 Pi{%) 

Money 

County per Capita Income (CCI) -.061 .004 OA- 
Average Professional Salary {APS) .236 .056 15^61 
Expenditure per Pupil (EPP) -.003 .00 0.0 
School System 

Average Daily Membership (ADM) -.166 .028 2.8 

Percent Student Attendance {°/<^A) .330 .109 110.91 

Percent Oversized Class (%0C) .076 .006 0.6 

Percent Free/Reduced Lunch {%fF{L) .244 .060 I 6.01 

Percent Career Ladder II & III (%CL) -.040 .002 0.2 

TOTAL — — 26.5% 



Significant negative correlations occurred between the following eight factors: 



a. CCIand%FRL r=-.53 

b. APS and %0C r= -.46 

c. EPPand%OC r=-.39 

d. APSand%FRL r=-.37 



e. CC!and%OC r=-.36 

f. %OCand%CL r=-.35 

g. %FRL and %CL r= -.32 

h. ADMand%FRL r=-.26 



Most of these correlations are not surprising. Several relattonships demonstrated in the negative comelattons 
reflect "common sense." Per capita irK»me and high % FRL are generally perceived to be related indicators of 
low socto-economic status of a community. Limited resources, higher nunfibers of free lunches, lower professional 
salaries, more oversized classes, and lower exf)enditures per student go hand in hand. Nor is it surprising that 
greater %CL teachers are found in communities with more money and lower class sizes. 
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Table 8. Profile of Z-scores of Nine Highest and Nine Lowest Ranked School Districts by 
MQO 2 score, TN; 1988-89 data. 



OUTCOME MONEY SCHOOL SYSTEM 

MSO CCI APS EPP ADM %SA %5C %FRL %CL 

n 121 121 121 121 121 121 110 121 121 



Tops 

RANK 



1 


1.97 


1.00 


2.18 


0.91 


-0.26 


0.92 


0.87 


1.30 


1.81 


2 


1.40 


1.44 


2.67 


3.64 


-0.18 


-0.20 


1.39 


0.53 


2.06 


3 


1.26 


-0.24 


-0.10 


-0.66 


-0.42 


1.79 


0.59 


0.46 


-0.90 


4 


1.20 


-2.00 


-1.14 


-0.29 


-0.47 


0.99 


0.81 


-1,39 




C 

O 


1 17 
1.1/ 




n Ri 

U.Dl 


ft 

U. 1 D 


u.oo 


u.oo 


n on 


P 1 1 


n fid 


6 


1.50 


-0.28 


-0.17 


-0.27 


-0.30 


-0.04 


-0.86 


-0.16 


0.38 


7 


1.14 


-0.13 


1.63 


2.87 


-0.31 


1.71 


1.34 


0.46 


-0.16 


8 


1.12 


-0.24 


-0.37 


-0.55 


-0.47 


1.79 


na 


-0.22 


1.62 


9 


1.11 


1.00 


3.66 


3.24 


-0.43 


1.31 


na 


0.60 


2.36 


M 




0.51 


1.00 


1.00 


-0.27 


0.99 


0.73 


0.41 


0.75 












Bottom 9 










113 


-1.04 


-0.78 


-0.71 


-0.45 


-0.32 


-0.42 


0.34 


-0.09 


-0.21 


114 


-1.17 


-0.35 


-0.60 


-0.54 


-0.19 


ML 


na 


-2.08 


-0.35 


115 


-1.29 


-1.07 


-1.31 


-0.93 


-0.40 


-0.36 


-2.65 


-0.02 


-1.33 


116 


-1.33 


-0.47 


-0.98 


-0.10 


-0.14 


-1.00 


0.39 


-1.60 


-0.40 


117 


-1.43 


ZM 


IM 




7.92 


-Q.A4 


0.78 


-2.36 


0.16 


118 


-1.55 


-1.59 


-1.31 


-0.05 


-0.34 


-2.04 


-1.44 


-1.60 


-0.60 


119 


-1.77 


-1.03 


-0.55 


0.15 


0.02 


-2.92 


-0.05 


-1.53 


-0.94 


120 


-2.14 


-0.13 


■1.36 


-0.61 


-0.14 


0.01 


-1.12 


-3.18 


-1.75 


121 


-2.53 


-1.23 


-1.20 


-1.12 


-0.34 


-2.76 


-2.65 


-0.16 


-1.39 


M 




-0.51 


-0.74 


-0.27 


0.67 


-0.94 


-0.80 


-1.40 


-0.76 



Bold = MSO z-score and category z-score are similar (i.e., both are either positive or 

negative and are greater than one). 
Underline = MSO z-score and category z-scores are not similar (i.e., if one is a ±1 or larger SD, 

the other item is a zl or larger SD) . 



3. When rank ordered on the basis of student outcomes, do school districts within the 
state pert onn as expected in temis of the money and school system characteristics 
reported in the Report Cards ? 

One might expect that school districts with more money, better teacher salaries, enrollments which 
are not extreme (very large, very small), fewer oversize classes, higher attendance and high numbers of 
upper-level Career Ladder teachers would produce the best MSO. In general, those expectations were 
confirmed in the regression analyses conducted in response to research question one. There are, 
however, outlier districts producing MSOs significantly lower or higher than could be expected. Figures 
9 and 10 (p. 14) show these exceptions. Tables 8 (p. 17) and 9 (p. 18) highlight these exceptions. 
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Table 9. Profile of ranks on eight variables of the nine highest and nine lowest ranked (Rk) school 
districts by MSO z-score, TN: 1988-89 data. 



MONEY SCHOOL SYSTEM 



Rk Rk Rk Rk Rk Rk Rk Rk 

RANK CCI APS EPP ADM %SA %0C %FRL %CL 

n= 121 121 121 121 121 110 121 121 



1 


14 


6 


2 


12 


3 


3 


m 


59 


4 


m 


m 


5 


1 


27 


6 


n 


61 


7 


66 


8 


8 




Zl 


9 


14 


1 


113 


99 


90 


114 


80 


82 


115 


105 


120 


116 


88 


105 


117 


2=5 


13 


118 


117 


119 


119 


103 


80 


120 


63 


121 


121 


108 


115 



Top 9 



18 


M 


18.5 


1 


50 


Z2 


M 


104 


6.5 


S& 


m 


16.5 


33 


11 


26.5 


64 


74 


65 


6 


za 


8 


S5. 


m 


6.5 


2 


108 


11.5 



Bottom 9 



78 


80 


84.5 


84 


51 


a 


111 


101 


80.5 


^ 


M 


106 


13 


1 


84.5 


SZ 


84 


117 


24 


22 


121 


90 


45 


60.5 


116 


87 


120 



24 


5.5 


6 


1 


39 


5 


36 


44.5 


103 


26 


106 


IflZ 


18 


1 


27 


S2 


M 


37 


2 


44.5 


60 


na 


87.5 


8 


na 


33 


3 


M 


79 5 


62 


na 


116 


74.5 


107 


74 


114 


42 


112.5 


78.5 


27 


118 


42 


101 


112.5 


88.5 


64 


109 


104 


97 


120.5 


119 


108 


84 


116 



U nderlined = a 60 ranking difference between the district's MSO rank and the category rank. 

In Table 8, the nine highest and nine lowest ranked districts by MSO are profiled by their z-scores on 
the eight independent variables. The district producing the fourth highest MSO does not conform to 
expectations in at least four of eight areas. It has low CCI and APS, a high %FRL and a small %CL. It 
does not rate as high as other districts in the "top nine" category in several other areas. It would not be 
expected to be a *1op nine" MSO school district, but it is. 

The school district ranked number 117, fifth from the bottom in MSO in the sample of 121 school 
districts is also an "outlier." It has the second highest CCI z-score (2.04) for the 1 21 school districts, high 
APS, and large EPP as denfX)nstrated by rank and z-scores. It is, however, a large school district as 
demonstrated by its z-score in that category (7.92). 

Table 9 (p. 19) data are also useful in responding to research question #3. The nine highest 
performing school districts (MSO) and the nine lowest performing distrkJts were profiled among all school 
districts studied based on their ranks on the eight independent variables. 
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The positional profile reveals that the third, fourth, sixth, and eighth highest performing school districts 
do not conform to commonly held beliefs atxxjt scf ool district perfomiance (e.g., a direct, positive 
correlation between MSO and school system data). District ^ places 73rd in CCI among 121 districts and 
94th in EPP. It is a small (104th ranking), rural school district -/ith relatively few Career Ladder teachers 
{ 103rcl ranking . However, it has proportionately high %SA [tied for 6th position) among 1 21 districts. 

School district #4 has the lowest per capita income among districts in the sample ( I2lst rankinsfj. It is 
1 1 0th in APS and 66th in EPP. It is also a small, rural school district (1 18th rankin(i) with high %FRL ( 106th 
position) and low %CL teachers ( 107th position). However, it is tied for 16th place %SA and only 2eth in %0C, 
indicating fewer oversized classes than roughly four-fifths of the districts studied. The eighth highest performing 
school district demonstrated a profile similar to district #4, except that its attendance was better (tied for sixth), 
and it had a higher %CL ( eighth position). 

The school district with the eth highest MSO consistentl> ranked in the lower half of all districts on ail factors 
except %CL teachers where it ranked at the bottom of the upper quartite. 

The MSO factor in all of these districts is relative. These top nine are the best performing districts in the 
state, but the state averages typically do not place TN schools avnoog the best perfomners in the countfy. It is 
clear, however, that money id not the most critical factor in MSO in TN's "high flyers." If any factors seem more 
ctosely associated with MSO more than any others, they are school district size (ADM) and oversized classes 
(%0C) as negatives and attendance (%SA) as positive. However, there are districts that do not have the 
highest %SA or lowest %0C that outperform the majority of districts in the state. Other factors, not present in 
these report cards but associated with student arJiif^v ftment. are operating. These findings interface with the 
finding that the eight factors studied account for only 26.5 percent of the mean student outcome. 

The analysis of the nine worst-perfomiing districts confinrc the findings on the other end of the spectrum. 
Some profiles among this group do not conform to commonly heW beliefs or even to the patterns of the "outlier" 
schools with top perfonriance rankings. District #1 1 7 ranks among the top districts in the state in all three 
money factors, in the top quartile in oversized classes (small numbei), and in the top half in %CL. It has high 
%FRL ( 1 18th position) and is fifth from the last in MSO. The factors that might contribute to tow performance 
are the large size of the school district ( 1st ranking, low student attendance (ranks in the bottom third of 
districts) and tow SES ( 1 18th in free/reduced lunches). However, other districts with similar features (except 
size) did far better in student pertomnance (e.g., #4, except for %SA). 

The findings in response to research questton #3 present a mixed message. Many districts did not perfomi 
as one could have expected based on the factors studied. They support the notfon that the factors studied are 
not the primary determiners of student achievement. 
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4. Do the reported school district characteristics appear to represent most ^ors which relate to 
student academic achievement? 

The answer to this question is clearty "no " The partial correlation and percentage of influence analyses 
reported in response to research question #2 (see Tables 6 and 7, p. 16) indicated that the eight factors studied 
as independent variables accounted for only one fourth of the influence on MSO. The ranking analysis for 
research question #3 also suggested that in both the highest and lowest performing school districts, factors 
other than those reported in the report cards were at work. Almost three-fourths of whatever accounts for 
student academic performarx;e (as measured here by MSO) is unaccounted for based on these report cards- 
apparently in contradiction to 'lolk wisdom" about what determines student achievement. 

5. When academic achievement Is treated as scores on three separate test batteries, are patterns of 
Influence changed? 

Table 10 (p. 21) presents the results of Guttman's partial conelation procedure applied to the relationship 
between each independent variable and the three separate mean scores (Basic Skills First, Stanford Test of 
Academic Skills, Tennessee Proficiency Test) reported by each school district. A nxjre detailed presentation of 
these data appear as Table 1 1 , expanded in the Appendix B (p. 33) . 

The combined MSO has been used throughout the study because it seems the strongest measure of a 
school district's student achievement. All three mean test scores reported by a district are unique and separate 
measures. They are used for different decisions by Tennessee polfcymakers. In representing several tests and 
grade-level results, the MSO statistic seems to represent total student outcome best. However, investigators 
wished to see if influence patterns differed significantly when each available outcome measure was treated 
separately. 

The combined eight factors used as independent variables account for only 18.6% of wfiatever influences 
scores on the BSF tests, 1 4% percent of the influence on the STAS scores, and 1 7.2% of TPT results. In other 
words, the influence of these eight "independent" variables varies by test, and no pattem of influence is greater 
than that reported for the combined MSO. Further, these variables influence scores on state devek>ped tests to 
a greater extent than nationally developed tests. 

When reviewing the relationships between the several independent variables and student outcomes, it is 
interesting to note that the variable exerting the greatest influence on student outcomes, attendance (%SA) 
does not influence scores on the STAS battery at all (0%), but does influence BSF (10%) and TPT (12%). 
Percentage of free or reduced funchos does not influence any single test score more than 4%, but that factor 
accounts for 6% of the influence on combined MSO. The %CL in a school district appears to influence only 
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4. Do the reported school district characteristics appear to represent most factors which relate to 
student academic achievement? 

The answer to this question is clearly "no." The partial con-elation and percentage of influence analyses 
reported in response to research question #2 (see Tables S and 7, p. 16) indicated that the eight factors studied 
as independent variables accounted for only one fourth of the influence on MSO. The ranking analysis for 
research question #3 also suggested that in both the highest and lowest performing school districts, factors 
other than those reported in the report cards were at work. AlnfX)st three-fourths of whatever accounts for 
student academic performance (as measured here by MSO) is unaccounted for based on these report cards- 
apparently in contradiction to 'lolk wisdom" about what detemiines student achievement. 

5. When academic ?^hievement Is treated as scores on three separate test batteries, are patterns of 
Influence changed? 

Table 10 (p. 21) presents the results of Guttman's partial conelation procedure applied to the relationship 
t>etween each independent variable and the three separate mean scores (Basic Skills First, Stanford Test of 
Academic Skills, Tennessee Proficiency Test) reported by each school district. A more detailed preservation of 
these data appear as Table 1 1 , expanded in the Appendix B (p. 33). 

The combined MSO has been used throughout the study t>ecause it seems the strongest measure of a 
school district's student achievement. All three mea i test scores reported by a district are unique and separate 
measures. They are used for different decisions by Tennessee policymakers. In representing several tests and 
grade-level results, the MSO statistic seems to represent total student outcome t)est. However, investigators 
wished to see if influence pattems differed significantly when each available outcome measure was treated 
separately. 

The combined eight factors used as independent variabtes account for only 18.6% of whatever influences 
scores on the BSF tests, 14% percent of the influence on the STAS scores, and 17.2% of TPT results. In other 
words, the influence of these eight "independent" variables varies by test, and no pattern of influence is greater 
than that reported for the combined MSO. Further, these variables influence scores on state developed tests to 
a greater extent than nationally developed tests. 

When reviewing the relationships between the several independent variables and student outcomes, it is 
interesting to note that the variable exerting the greatest influence on student outcomes, attendance (%SA) 
does not influence scores on the STAS battery at all (0%), but does influence BSF (10%) and TPT (12%). 
Percentage of free or reduced lunches does not influence any single test score more than 4%, but that factor 
accounts for 6% of the influence on connbined MSO. The %CL in a school district appears to influence only 

20 



ERIC 



23 



Table 10. Guttman*s Partial Correlations Between Independent and Dependent variables 

Total 





CCI 


APS 


EPP 


ADM 


%SA 


%oc 


%FRL %CL 


% 

Influanca 


BSF 


0.5 


0.7 


1.3 


1.3 




1.1 


3.4 


0.2 


18.6 


STAS 


0.1 


7.0 1 


0.0 


2.9 1 


0.0 


0.5 


3.6 


0.0 


14.0 


TPT 


0.1 


2.3 


0.1 


0.5 


12.2 


0.0 


2.0 


0.0 


17.2 


MSO 


0.4 


5.6 


0.0 


LmJ 


1Q,-9 


0.6 


1 6.0 


1 0.2 


26.5 



Box = Any percentage of influence rounded to, or larger than 3.0. 

scores on the BSF, and that influence is almost negligible (0.2%). The greatest influence on Stanford scores 
among variables reported is APS (7%). Class size (%0C) inlluences only the BSF test scores at a level 
representing 1% or less of the influence exerted. Finally, APS, ADM, %SA, and %FRL appears to influence to 
some degree student outcomes, while CCI, EPP, ^.lOC. and %CL have negligible influence on outcomes. 

The data from this analysis confirm that combined influence of the eight factors reported is relatively small. 
There is much more that needs to be known about what contributes to student achievement. TTiey also suggest 
that different tests may be influenced differently by the sanr^ factors, and that there may be need to consider the 
importance of differences in assessment measures produced at/for a state level and those produced at/for 
natbnal use. 

6. Do "outlier" school districts signtf Icantly influence the findings of this study? 

As described in the Methodology section of this paper, two procedures were used to develop answers to 
question #6. Rrst a truncated MSO correlation coefficient was established for the interaction of each 
independent variable with the MSO of the "middle" 97 school districts. The 97 school districts used in this 
computatton were detemfiined by eliminating the top 1 0% and bottom 1 0% of school districts on the ranking 
continuum for each independent variable. Here, the data were truncated to fulfill the Pearson Product Moment 
con-elation assumptions of bivariate normal distribution and homoscedasticity. This alternative statistical 
analysis was used to con^are the "r" with the Polynomial regression "r". A second step in both procedures was 
application of Fisher's Zr-transformatbn to determine the significance between the original con-elation 
coefficients and the coefficients produced by the newly applied procedures. 

The results of the truncated MSO procedure are presented in Table 1 1 and Figures 11-18 (Appendix C, 
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Table 1 1 . Differences between MSG and Tmcated MSO cx^n-elations and eight Tennessee Report 
Card categories, 1989 Tennessee Report Card data. 



AREA MSO MSO Differonco 

Truncated 

n=121 n^97 





r 


z 


r 


z 


r 


z 


t 


CCI 


.313 


.321 


.421 


.448 


.108 


.127 


.919 


APS 


.436 


.466 


.228 


.229 


.208 


.237 


1.714* 


EPP 


.223 


.224 


.084 


.084 


.139 


.140 


1.013 


ADM 


-.090 


.090 


-.032 


.032 


.058 


.058 


.420 


%SA 


.407 


.430 


.210 


.213 


.197 


.217 


1.570* 


%0C 


-.302 


.310 


-.126 


.127 


.176 


.183 


1.323* 


%FRL 


-.488 


.530 


-.288 


.293 


.200 


.203 


1.468* 


%CL 


.310 


.321 


.200 


.203 


.110 


.118 


.854 



p<10 
p<05 



pp. 34-37). These findings are presented in the foltowing paragraphs with brief explanations of their contents as 
compared to the findings reported for research question #2. 

Table 1 1 depicts significant relationships between the oonrelation ooefffcients established for four variables: 
average professional salary, percentage of student attendance, percentage of oversize classes and percentage of 
students on free and reduced lunch. Only one of these interactions was significant at the .05 level: Average 
Professbnal Salary (APS). 

The impact of the deletion of "outlier" school districts from consideration is shown in the figures presented in 
Appendix C (pp. 34-37). Trendlines generated in these figures can be compared with those displayed in 
Figures 1 through 8 on pages 29 through 32. 

Figure 1 1 (p. 34) displays a positive correlation between CCI and MSO, when the truncated MSO 
procedure is applied. Comparison of Figure 1 1 with Figure 1 (p. 29) shows their similarity. The con'elatton and 
resulting trend line are sinrply not as pronourxed (steep) wfien the counties with highest and lowest CCI are 
removed from consideration (r=.313 , 421 , respectively ). 

Figure 1 2 (p. 34) displays a positive correlation t)etween APS and MSO, as did Figure 2 (p. 29) in the initial 
analysis. The deletion of outlier districts decreases the slope of the trend line markedly . The difference (p<.05) 
between the con'elation coefficients (r=.436,.228, respectively) produced in the original analysis and the 
truncated MSO analysis (Table 1 1 ) suggests that salaries in the outlier school districts "drove" the degree of 
relationship found between these variables in our first analysis. 

Figure 13 (p. 35) should be compared with Figure 3 (p. 30). Threre is a positive relationship between EPP 
and MSO. However, the truncated MSO analysis levels the trend line considerably. Outlier school districts 
spending much smaller or much greater amounts of money than the majority of districts influence the 
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relationship of variables reported in tiie first analysis. 

Figure 1 4 (p. 35) depicts the relationship of ADM to MSO using the truncated MSO analysis. When 
conrparing the trendline in Figure 14 to that in Figure 4 (p. 30), very small and/or very large districts in the state 
accoumed for all the negative con-elation found in the first analysis. Apparently, ADM influences student 
outcomes most negatively when a school district is large. However, neither congelation was significant. 

Figure 15 (p. 36), like its counterpart, Figure 5 (p. 32) displays the relationship between %SA and MSO. 
The significant {p<.1 0) relationship is positive in both analyses, but the outlier school districts had a profound 
effect on the stope of the trendlines. It is difficult to conclude exactly the relationship of %SA to student 
outcomes by studying these two figures. 

The effects of %0C on MSO is re-examined in Figure 16 {p.36)(For comparison, see Figure 6, p. 31). 
While the truncated MSO analysis yields a slight negative relationship, the trendline in the Initial analysis 
containing outlier school districts was much steeper. Comparison of the initial points on the lines demonstrates 
only small differences. A small %0C does not effect cun-ent student outcomes in a school district greatly. A 
large %0C does have substantial negative impact. 

Results of the tmncated MSO analysis of the impact of %FRL are displayed in Figure 17 (p. 37). The 
comparable figure in the original analysis Is Figure 7 (p. 32). In both analyses, greater %FRL correlate 
negatively with MSO as reported in Table 1 2. Negative relationships were reduced significantly when outlier 
school districts were rerrvoved from the analysis. 

Figures IB (p. 37) and 8 (p. 32) portray the impac. of %CL on MSO. In both "portraits", the relationships 
are positive; i.e., the greater the %CL, the higher the student outcorr)es. 

Several findings resulting from the truncated MSO analysis of available data lead to the finding that the 
outlier school districts (top 1 0% and bottom 10%) as defined for each independent variable substantially impact 
the relationship between that variable and MSO, not in the direction of the relationship, but in its intensity. 

C. Results of the Polynomial Regression Analysis 

A second procedure was applied to available data to try to answer the question of outlier school districts' 
impact on the relationships between independent variables and the dependent variable MSO. Table 12 
displays the results of computation of a 3rd order cubic polynomial for each relationship as oonnpared with the 
Pearson Product Moment conclation coefficient developed in the first analysis. In only two categories, %0C 
and %CL, were differences in the two coefficients significant at the .10 level. However, Rgures 19 through 26, 
Appendix D (pp.38-41), portray aspects of the various relationships which were not generated by the statistical 
treatments used in the first two analyses. 

The polynomial analysis of the relationship between CCI and MSO displayed in Figure 19 (p. 38) clarifies 



26 



the positive correlalion between tlie variables. Differences between CCI of $5,000 and $7,000 appear to have 
little effect on student outcomes. Increases in CCI between $7,000 and $17,000 influence student achievennent 
positively. 

Figure 20 (p.38) shows n^ltiple aspects between APS and MSO. Increases in APS between $20,000 and 
$26,000 generally contrilxjte to positive increases in student achievement, increases between $26,000 and 
$31 ,000 appear to have relatively little additional impact on student achievement, but when average salaries 
move beyond $32,000, the positive relationship rises dramatically. The APS prot)ably reflects the experience of 
professional staff as well as connpensation. The figure may indicate that an experienced professional staff 
generates higher student achievement than does an inexperienced one. 

The 3rd order polynomial analysis of the relationship between EPP and MSO (Figure 21 , p. 39) produces a 
trend line closely resembling a straight line relationship. Increases in EPP positivefy correlate with MSO at all 
points along the line; however, the inrpact of increases in these expenditures is nrx^re noticeable between $2,000 
and $3,000 and between $4,000 and $5,000 than between $3,000 and $4,000. 

Figure 22 (p.39) presents the po^ynomial analysis of the relationship t>etween ADM and MSO. There is a 
slight positive interaction of these variables with increases in size up to 40,000 students. This positive 
correlation may reflect the ability of a somewiiat larger school district to provide curriculum breadth and more 
instructional resources. Increases in size between 40,000 and 60,000 students do not seem to change the 
relationship. In the on^ school district beyond 60,000 students, student achievement decreased. 

Figure 23 (p. 40) shows the relationship between %SA and MSO, using the 3rd order cubic polynomial 



Table 1 2. Fisher's Zr-transformation was used to compare the Pearson Product Moment "r*' with the 3d 
order "Cubic" polynomial regression "r**. 

AREA MSO MSO Difference 

3rd order "Cubic "Polynomial 
n=l2l n=l2l 





r 


z 


r 


z 


r 


z 


t 


CCI 


.313 


.321 


.344 


.354 


.031 


.033 


.320 


APS 


.436 


.466 


.508 


.556 


.072 


.090 


.873 


EPP 


.223 


.224 


.290 


.299 


.058 


.075 


.727 


ADM 


-.090 


.090 


-.192 


.192 


.102 


.102 


.990 


%SA 


.407 


.430 


.330 


.343 


.109 


.087 


.843 


%0C 


-.302 


.310 


-.446 


.478 


.199 


.168 


1.629* 


%FRL 


-.488 


.530 


-.500 


.549 


.012 


.019 


.184 


%CL 


.310 


.321 


.434 


.464 


.124 


.143 


1 .386 * 



* p<.10 
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analysis. Increasing attendance correlates positively witli student achievement, until attendance reaches about 
96%. Inaease from 96% to 97% does not appear to change the relationship. Beyond that point there is a ni?nht 
downturn in the relationship. Further, 99% or 1 00% attendance would mean that the least'abte leamers in tlie 
district were in school most of the time. This phenomenon might result in lower district wide test scores. 

Figure 24 (p. 40) shows a complex relatfonship between %0C and MSO. As the percentage of oversized 
classes increases from 0% to 18%, student achievement decreases. 

Figure 25 (p.41 ) presents the polynomial analysis of the impact on MSO of %FRL. When fewer than eight 
percent of the student body received this aid, achievement was not af ected. As the percentage rises from 8 to 
«85, achievement shows a significant downward trend. At that point the trend line '^ttoms out," and 
achievement actually begins a slight upward trend when 85% or more of the students in a school district receive 
this service. 

The cubic polynomial analysis of the relationship between %CL teachers in a school district and MSO is 
depicted in Figure 26 (p. 41 ). An increasing correlation is cleariy seen between the two factors as %CL rises 
from .0 to 25%. Between 25 and 50 percent, there is a dramatic downturn of the trend line due to the influence 
of one "outlier school system. Without this one district, the projected trendline would be level. 

V. CONCLUSIONS AND IMPLICATIONS OF THE STUDY 

Investigators developed a fifth research question as a means of trying to franie conclusbns and 
implications in a useful way: 

7. How might the findings of this study Infomi educational policy and policymakers at the state and 
local levels? 

a, Imputs Arent Enough . Several points seem clear. "Good" inputs to the edunational process {money, small 
classes, good community. e\c.) do not guarantfte high student achievement. If we must study student 
achievement, we might study outputs (outcomes) not inputs and, following "effective school" methods, try to 
determine "correlates". Inputs commonly assumed to be associated with outputs account for only minor 
variation in performance outcomes {just over one-fourth). The individual assodation of most of these factors 
(with the exception of attendance) is minor; debate over and attention to any one in isolatton may be a wasteful 
activity. If we want to innpact student performance, a comprehensive approach is needed. 

In recent years, there have been attempts to compromise comprehensive educational refc /1 packages by 
putting major emphasis and resources into one or two strongly lobbied dimensions of tJie conrprehensive plan. 
While this approach may soothe feelings, it will do little to improve student achievement. 



b. These Eight Inputs Don^ Explain Outcomes. Another cx)ndusion erf the study: the TN Rep^ 
deal wthard report many import Whatanethey? 

On the other hand, we may know more than we 're measuring and reporting, ft is currently popular to talk 
about restaicturing schools, teaching and learning conditions and parent-school relationships. Some educators 
suggest and have some research ("soft data") that student motivation and self-concept, student and educator 
attitudes, school climate, teaching strategies, administrator and teacher styles and parent involvenf)ent in 
schools must change before much change will be seen in student academic achievement. Those factors can 
be rather easily measured and reported. Note the RJR Nabisco Foundation's Next Century Schools Project, 
the Coalition of Essential schools and other such endeavors. How much influence do some of these other 
factors have on MSO? 

A major danger is that teachers and administrators may focus upon the "factors" reported on Report Cards 
to make their schools look good. Meanwhile, the important elements are overlooked. 

a Assessments Used May Be a Factor, Assessing what students know and are able to do is a problem. 
The investigators used available, reported test results in these analyses; the findings are only as good as the 
test results. There is debate about the value of present tests— both their content and fomiat (see Kacpao, Nov. 
1991). if the tests used to fomn the dependent variable in this study (MSO) were not measuring the right things 
or were not measuring them validly and reliably, all other findings reek. The investigators do not challenge any 
of the tests currently in use in Tennessee. One statistical eccentricity noted was that correlations between the 
factors studied and MSO were usually stronger wiien the single outcome measure of achievement was a 
national test: The Stanford Achievement Test, Task II, rather than a stale-specific test -^he Basic Skills or 
Proficiency test (BSFor TPT). 

Further, we cani ignore the relationship between teaching and testing. We have no information about the 
instruction and curriculum in these school districts. One would like to know those factors that conrelate with the 
test resufts. 

The use of the three analysis procedures in response to research questbn #6 also yiekJs inportant 
conclusions and implications. Data from "outlier" school districts influenced the state profile greatly. The same 
situation probably exists when profiles are created at a district level. "Outlier" schools within the district will 
influence the district profile. For both accountability and for school improvement purposes, building-ievel data 
are probably the most useful and the most valid data available. 

There app ear to be few, if any, "straight line" relationships between inputs and outcomes. Relationships 
between money expended and student achievemerit, between student attendance and student achievement or 
any other con^bination of variables are complex. They must t>e studied carefully. The "outlier schools also 
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offer addittonal food for thought. The data in this study suggest that individual schools and school districts have 
overcome the presence of negative connmunity and resource factors. We nnust determine how they have been 
able to do that, so that their experiences can be replicated. We should chaliege the propQsrtion that student 
outcomes are pre-determined by existing conditions. 

Analysis of Appendix B shows thai criterion-referenced test results (BSF, TPT) are influenced by %SA, but 
that the rwnrvreferenced test results (STAS) are not influenced by %SA. Additionally, the STAS is influenced 
by %APS (higher teacher salaries), while the criterion-referenced test results are not. This condition leads to 
some interesting speculatbn. 

These results should be considered tentative. This a study of relattonships; we make no claim for 
"generalizability," but we encourage people to consider similarities in these data and in report card construdton 
and use in their own situations. TTie release of The Report Card" may generate considerable attention-in 
smaller news martlets results may warrant front-page notice. Perhaps state educatton officials use report card 
results as a basis for deciding that a district is at risk, or even needs external help. What is included in the report 
card influences perception and policy. Some inclusions obviously can be nrasleading. 

In spite of their tentative nature, these findings are instructive. At least based on the results for this study, 
the media hyperbole and hoopla surrojnding report cards are vastly over-rated as current report cards deal with 
an underwhelming part of schooling. If you wanted to know all about somethirvg ( ^00%) and some strategy 
provided you 25% of the infomiatton, would you continue to emphasize that 25%, or would you want to find the 
missing 75%? Would you base reform and restructuring of education on what is known (25%) or would you 
seek to break the vnoki with new kleas? 

Finally, there is a note for those who are contemplating, creating or modifying report cards on schools. If 
the entries on those report cards are similar to those investigated in this study, they provide interesting statistics 
but only a small arTX)unt of informatton which can be of use in improving educatton. Informatton entered into the 
report card is of minimal use until additional analyses such as those conducted in this study are performed to 
seek out what data may be (or may not be) saying. 
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APPENDIX A 

Ungar Pggrgsslopg (n=i2i) 

Figures 1 through 8: Relation of Independent variables to student outcome (MSO) 

When reviewing the following figures, the reader should be aware of two aspects of each visual image. The 
black dots represent data from each school district included In the study. A visual survey of the figure will establish 
where and how the school districts cluster in the analysis. The direction of the trend line gives the reviewer an 
immediate indication of the nature of the correlation. Lines moving upward from left to fight suggest positive 
correlations. "Flat" lines indicate little or no correlation, and lines moving downward from left to right indicate 
negative correlations. The degree of the slop of the line ^ives good indication of the significance of the correlation, 
even without the statistics themselves. 
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County per Capita Income (CCI) 

Relation of Per Capita Inconne to Mean Student Outconne (MSO). 
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Average Professional Salary (APS) 

Relation of Average Professional Salary to Mean Student Outcome (MSO). 
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Expenditure per Pupii (EPP) 

Figure 3. Relation of Expenditure Per Pupil to Mean Student OutcxJme (MSO). 
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Figure 5. Relation of Percentage of Student Attendance to Mean Student Outcome (MSO). 
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APPENDIX B 
Guttman's Partial Correlations 
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APPENDIX C 
Truncated Li near Analysis (n=97) 

Figures 11 through 18: Relation of Independent variables to Mean Student Outcome (MSO). 
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Figure 16. 
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APPENDIX D 
Polynomial Regression Analysis (n=i2i) 



Figures 19 through 26: Relation of Independent variables to Mean Student Outcome (MSO). 
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Figure 19. 
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Figure 20. Percent Average Professional Salary (APS) to Mean Student Outcome (MSO). 
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Figure 23. 
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Figure 24. Percent Oversized Class (%0C) to Mean Student Outcome (MSO). 
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Relationship of Percent Free or Reduced Lunch (%FRL) to Mean Student Outcome (MSO). 
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